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Developing a Phonological Awareness Curriculum: 
Reflections on an Implementation 
Science Framework 
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Purpose: This article describes the process of developing 
and implementing a supplemental early literacy curriculum 
designed for preschoolers demonstrating delays in literacy 
development. 

Method: Intervention research and implementation 
research have traditionally been viewed as sequential 
processes. This article illustrates a process of intervention 
development that was paralleled by a focus on implementation 
in early childhood settings. The exploration, preparation, 
implementation, sustainment framework is used to 
describe factors that need to be considered during a 
progression through these 4 phases of implementation. 

A post hoc analysis provides insight into a rather nonlinear 


progression of intervention development and highlights 
considerations and activities that have facilitated 
implementation. 

Conclusions: The guiding principles of the exploration, 
preparation, implementation, sustainment implementation 
science framework highlight the important considerations in 
developing effective and practical interventions. Considering 
implementation and sustainment during the intervention 
development process and using data-based decision 
making has the potential to expand the availability of user- 
friendly evidence-based practices in communication 
sciences and disorders and encourage a bridging of the 
researcher—clinician gap. 


implementation research may be useful in elucidat- 

ing how the former is driven by client outcome 
data and the latter is driven by adopters’ use and percep- 
tions of the intervention. Intervention research generates 
knowledge on the effects, efficacy, and effectiveness of 
an intervention that typically progresses iteratively, ideally 
with single-case experimental designs scaling up to random- 
ized control and cluster randomized designs (Olswang, 
1990). Implementation research focuses on how to facilitate 
the process of embracing and adopting evidence-based 
practices and sustaining high-fidelity use in everyday set- 
tings (Fixsen, Naoom, Blase, & Friedman, 2005). 

The Center for Response to Intervention in Early 
Childhood (CRTIEC), a cooperative agreement funded by 
the U.S. Department of Education’s Institute of Education 
Sciences, had a unique charge. The research team was 
charged with developing practical tools or resources neces- 
sary for implementing a viable response to intervention 


r | 1 he distinction between intervention research and 
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(RTI) system to promote school success among children 
who are at risk for reading disabilities. The tools and 
resources the CRTIEC team sought to develop included 
effective curricula, screening and progress monitoring 
measures, a decision-making framework, and associated 
training materials needed for implementing an RTI model. 
According to the cooperative agreement, the research 
team was expected to 


. develop and evaluate Tier 2 and Tier 3 curricula for 
preschool language and early literacy skills, 


. develop and validate an assessment system for univer- 
sal screening and monitoring progress aligned with 
these curricula, 


. carry out supplemental research responsive to the 
needs of early childhood educators and policy makers, 
and 


. provide outreach and leadership to facilitate dissemi- 
nation of efficacious tools. 


The major focus of this article is on the development 
and implementation of the Tier 2 curriculum focusing on 
phonological awareness and alphabet knowledge. An over- 
view of the overall project results is provided in Greenwood 
et al. (2014). 


Disclosure: The authors have declared that no competing interests existed at the time 
of publication. 
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Generating a feasible solution to address this charge 
within the context of early childhood education presented 
several challenges. Early childhood education encompasses 
a vast array of settings; we focused on public school pre-K, 
Head Start, and child care center classrooms. When work- 
ing with this population, one must accommodate con- 
siderable diversity among children, who enter preschool 
with vast variations in their home language and literacy 
exposure. Teachers in early childhood settings vary consid- 
erably in their commitment (high rates of turnover) and 
professionalism, in their training (ranging from high school 
to master’s degrees), and often in their philosophies of what 
they deem appropriate instruction for preschool children 
(Barnett, 2003; Early et al., 2007; Lee & Ginsburg, 2007). It 
is difficult to conceive of a preschool teacher being able to 
provide high-quality general classroom instruction (Tier 1) 
as well as small-group instruction for children falling be- 
hind (Tier 2) and individualized instruction for children 
with significant needs (Tier 3) without considerable assis- 
tance. Thus, a feasible solution required a measurement 
and decision-making system and curricula that were well 
developed, effective, and most important, easy to implement. 

The genesis of our evidence-based approach to a 
Tier 2 early literacy curriculum was an intervention that 
embedded phonological awareness instruction on rhyming 
and alliteration in a small-group storybook reading session. 
Ziolkowski and Goldstein (2008) replicated the effects of 
this embedded intervention in multiple-baseline designs 
across behaviors with 13 low-income preschoolers with lan- 
guage delays. This early success led us to believe that we 
could translate this approach into an intervention that 
could be implemented with ease by paraprofessionals (e.g., 
instructional aides) in early childhood settings. As described 
below, much intervention research was required to achieve 
that goal. In addition, the focus on the acceptability and fea- 
sibility of implementation of the intervention informed the 
process from the outset. Although our explicit knowledge 
of implementation science was minimal at the outset of the 
project, it is instructive to present a post hoc analysis of 
the implementation process to identify what was facilitative, 
what may have been lacking, and what remains to be done 
as we continue to scale up our research on RTI in early 
childhood. 


Implementation Science Framework 


A number of frameworks have emerged within the 
field of implementation research. Many of these frame- 
works focus on the implementation and sustainment of cur- 
rently developed evidence-based practices. For example, 
Damschroder et al. (2009) developed the consolidated frame- 
work for implementation research, which is a metaframe- 
work that is based on current research in implementation 
science. Although this framework addresses intervention 
characteristics, it does not address the development process 
of the intervention itself. However, the exploration, prepa- 
ration, implementation, sustainment (EPIS) implementation 
science framework (Aarons, Hurlburt, & Horwitz, 2011) 


focuses on implementation as a process that may begin 
before an intervention has been fully developed. This frame- 
work best aligned with the development and implementation 
of our Tier 2 phonological awareness intervention. 

Aarons et al. (2011) described the EPIS framework 
with four phases of implementation: exploration, prepara- 
tion, implementation, and sustainment. In the following 
sections, we discuss the overall implementation process, 
describe strengths and weaknesses of the process, reflect on 
ways to improve the implementation and sustainability 
of the intervention, and discuss areas for future research on 
implementation of early literacy interventions. 

The EPIS model was developed as a conceptual frame- 
work for implementing evidence-based practices in public 
service settings. The framework was specifically developed 
with child and family public service programs in mind 
(Aarons et al., 2011). Although our research team was not 
familiar with this work during the exploration phase of im- 
plementation, in retrospect, the process our team followed 
mirrored the major tenets of this model. Similar to other 
implementation models, there are inner and outer contexts 
to consider. This framework is unique in that variables 
within the inner and outer contexts influence each phase of 
the model. The EPIS model embraces iterative develop- 
ment, recognizing that the progression through these phases 
is not always linear, as earlier phases may be revisited dur- 
ing the development process. 

Table | illustrates factors typically addressed at each 
phase of the implementation process in public sector ser- 
vices. The conceptual model considers outer context factors 
and inner context factors. These inner and outer factors are 
interconnected, and the characteristics of the innovative 
practices are evaluated for goodness of fit within organiza- 
tions and systems. The outer context includes an evaluation 
of the broader interorganizational environment, the service 
environment, and consumer support. The inner context in- 
cludes an evaluation of organizational characteristics and 
individual adopter characteristics. Table 1 shows how these 
factors differ depending on the phase of implementation. 

In applying this model to preschool settings, we classified 
inner contexts as anything at the level of the classroom 
inward. For example, individual classroom, teacher, and 
child and family characteristics are considered inner con- 
texts. Outer contexts refer to factors broader than the indi- 
vidual building or classroom, such as federal and state 
policies, school district initiatives, administrative structures, 
and funding sources. 


The Intervention and Implementation Context 


The development of the supplemental phonological 
awareness curriculum seems to align with phases of the 
EPIS implementation science framework, but the progres- 
sion was not linear. Less than stellar effects during the 
initial implementation phase resulted in a return to ex- 
ploration and preparation using a second iteration of the 
intervention. Figure | includes the stages of intervention 
development and the corresponding phases of the EPIS 
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Table 1. Exploration, preparation, implementation, sustainment model illustrating factors that influence stages of implementation. 


Factor 


Exploration 


Preparation (adoption decision) 


Implementation 


Sustainment 


Outer context 
Interorganizational 
environment 


Service environment 


Consumer support 


Inner context 
Organizational 
characteristics 


Individual adopter 
characteristics 


Interconnections among 
and within organizations 


¢ Sociopolitical context and 
policies 

e Direct and indirect networks, 
technical assistance centers, 
professional organizations 

¢ Grant funding, continuity of 
funding 


e Accountability policies 
¢ Monitoring and review 


e Advocacy organizations 


e Absorptive capacity and skills 
e Readiness for change 

¢ Culture and climate 

e Leadership 


e Values and goals 
¢ Social networks 
e Perceived need for change 


e Innovation characteristics 
e Fit within system and 
organization 


¢ Sociopolitical context and policies, 
legislation, definition of evidence 

¢ Organizational linkages, leadership 
ties 

¢ Formal and informal information 
sharing 

¢ Funding tied to state and federal 
policies 


e Accountability policies 
e Monitoring and review 


¢ State or national advocacy 
¢ Lawsuits 


e Size 

¢ Role specialization 

¢ Knowledge, skills, and expertise 
e Values 


e Leadership 
¢ Culture embedding 
¢ Championing adoption 


e Innovation characteristics 
e Fit within system and 
organization 


¢ Legislative priorities 

¢ Administrative costs 

¢ Organizational linkages, 
leadership ties 

¢ Information sharing cross- 
discipline translation 

¢ Funding for training and 
sustained fiscal support 


¢ Intervention developers 

engaged in implementation 
¢ Effective leadership practices 
* Cross-level congruence 


¢ Community-based 
organizations 


¢ Structure 

* Goals and priorities 

¢ Receptive and ready for 
change 

¢ Culture and climate 

e Evidence-based practices 
structural and ideological fit 


¢ Demographics 

e Adaptability 

¢ Attitudes toward evidence- 
based practice 


e Innovation characteristics 
¢ Fit within system and 
organization 


¢ Sociopolitical leadership 

e Federal, state, and local 
policies and initiatives 

¢ Positive public-academic 
collaborations 

¢ Valuing multiple perspectives 


¢ Funding fits with existing 
service funds or absorptive 
capacity 

¢ Workforce stability 


¢ State and local organizations 


¢ Leadership 

¢ Embedded evidence-based 
practice culture 

* Critical mass of evidence-based 
practice provision 

¢ Culture and climate 

¢ Social network support 

¢ Fidelity monitoring support 

¢ Role clarity and staff selection 
criteria and practices 


¢ Demographics 

e Adaptability 

e Attitudes toward evidence- 
based practice 


e Innovation characteristics 
¢ Fit within system and 
organization 


From “Advancing a Conceptual Model of Evidence-Based Practice Implementation in Public Service Sectors,” by G. A. Aarons, M. Hurlburt, and S. M. Horwitz, 2011, Administration & 
Policy in Mental Health, 38, pp. 4-23. This is an open access article distributed under the terms of the Creative Commons Attribution Noncommercial License. 
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Figure 1. Phases of phonological awareness intervention development. 
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Implementation 


Cluster randomized design trial 


framework. Aarons et al. (2011) explained that the imple- 
mentation process often includes revisions and a return to 
early phases. The following sections describe the process of 
trials and revisions for optimizing the phonological aware- 
ness intervention. We conclude by reflecting on the process 
to determine how future interventions can be developed with 
wide-scale implementation a consideration from the start. 

During the exploration phase of implementation, 
outer context factors included obtaining grant funding and 
examining previous research and emerging policies on RTI 
related to early literacy skills. An implied policy mandate 
for RTI in early childhood, state and local education 
agency interest, and federal funding provided a supportive 
environment. Greenwood et al. (2012) highlighted many 
obstacles to overcome in implementing RTI in early child- 
hood settings. These challenges include inconsistent and 
often limited teacher preparation and resources, an absence 
of or poor implementation of evidence-based curricula, a 
lack of intervention strategies tailored to at-risk children, 
and varying levels of administrative support. Thus, any in- 
terventions developed by the CRTIEC research team would 
need to be user friendly and low cost to accommodate these 
barriers to RTI in early childhood. 

The inner context required the nurturing of interest 
and collaborative relationships with classrooms in the 
vicinities of our multisite research team in five states: Ohio, 
Kansas, Minnesota, Oregon, and Florida. The relationships 


varied on the basis of differences in service delivery models 
and goals among sites. For example, public pre-K in Ohio 
and child care centers in Florida were both motivated 

by the need to improve kindergarten readiness test results, 
but the classrooms differed in the training level of teachers 
and the time devoted to intentional instruction. Child and 
family characteristics were rather similar in that administra- 
tors directed our involvement to low-income neighborhoods 
in urban settings. 

Previous research has identified children from minor- 
ity groups and children with low socioeconomic status 
(SES) as being at greater risk for reading difficulties than 
nonminority, middle-class children (Snow, Burns, & Griffin, 
1998). Children from low-SES homes are at greater risk for 
learning deficits for early literacy skills, including phonologi- 
cal awareness, alphabet knowledge, functions of print, and 
oral language (Bowey, 1995; Hart & Risley, 1995; Lonigan, 
Burgess, Anthony, & Barker, 1998; Smith & Dixon, 1995). 
However, low-SES children have demonstrated favorable 
responses to early literacy interventions (Ehri et al., 2001). 
This previous research on risk and resiliency served as part 
of the theoretical foundation for the mission of CRTIEC. 
In keeping with the need to have high-quality general instruc- 
tion in an RTI model, we targeted receptive environments for 
implementation. Our minimal criteria for selecting partners 
for intervention development and eventual implementation 
were that classrooms were not currently implementing RTI 
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and had adopted curricula with a scope and sequence that 
included early literacy targets. 

Early in the CRTIEC project, the research team con- 
ducted a large descriptive study to explore the quality of 
typical Tier 1 instruction in preschools (Greenwood et al., 
2012). To better understand the need for RTI in early 
childhood settings, Greenwood et al. assessed more than 
600 children’s language and early literacy skills from 
65 classrooms across four states at the start and finish of 
the preschool year. At the start of the year, students were 
assigned to Tier 1, 2, or 3 on the basis of their performance 
on early literacy screening tools. All students received 
“business as usual” instruction in the classroom. Students 
who were identified as Tier 2 or Tier 3 candidates at the 
start of the school year failed to close the performance gap 
by the end of the year. This finding supported the need 
for developing supplemental early literacy interventions for 
early childhood classrooms. 

A theory of change informed our intervention re- 
search and guided the development and evaluation of a sup- 
plemental early literacy curriculum. The theory of change 
depicted in Figure 2 included inner contexts such as char- 
acteristics of the child, family, teacher, and classroom. 
According to the theory of change, students would range in 
risk and resiliency on the basis of these factors, as these pre- 
dictors were potential moderators of the effects of inter- 
vention. On the basis of the results reported by Greenwood 
et al. (2012), at-risk students demonstrating delays in early 
literacy skills who receive “business as usual” classroom ex- 
periences were likely to receive limited explicit instruction 
on early literacy skills and thus continue to perform below 
their peers on corresponding measures. We hypothesized 
that effective supplemental instruction with explicit instruc- 
tion, high rates of student engagement, repeated practice 
opportunities, and appropriate teacher feedback would pro- 
vide a solid foundation to foster growth in early literacy 
skills and help at-risk students close the achievement gap. 
Thus, this theory of change served as the foundation for the 
intervention development process. 


The Intervention Development Process 
Preparation of First Iteration 


The preparation phase of the intervention included 
redesigning the intervention to encompass a more complete 
scope and sequence of phonological awareness skills. In 
addition, we sought to coordinate our Tier 2 intervention 
for children with delays in early literacy skills with the de- 
velopment of a Tier 3 intervention for children with more 
serious delays (Kaminski, Powell-Smith, Hommel, McMahon, 
& Aguayo, 2015). Moreover, participant testing using stan- 
dardized measures, such as Test of Preschool Early Literacy 
(TOPEL; Lonigan, Wagner, Torgesen, & Rashotte, 2007), 
helped evaluate the adequacy of individual growth and 
development indicators (IGDIJ) for identifying children in 
need of tiered support (McConnell, Wackerle-Hollman, 
Bradfield, & Rodriguez, 2014). IGDI measures of rhyming, 


sound identification, first sound identification, and 
Dynamic Indicators of Basic Early Literacy Skills (DIBELS) 
Word Part Fluency and First Sound Fluency (Good & 
Kaminski, 2002) measures were general outcome measures 
that had the potential to sort children into appropriate tiers 
of support and monitor their progress with good accuracy. 

The Ziolkowski and Goldstein (2008) study embedded 
intervention in existing children’s books. However, the 
CRTIEC researchers determined that the time and cost of 
pursuing copyright permission for using published children’s 
books was a major obstacle. The alternative task of creating 
our own books was time consuming but offered a number of 
advantages: We were able to control the length, vocabulary, 
and language; generate rhyming text; create book series with 
recurring characters; and encompass themes common to pre- 
schoolers’ experiences. With the help of a professional nar- 
rator and professional illustrator, the team put together two 
series of prerecorded storybooks using Story Friends char- 
acters (Kelley & Goldstein, 2015). During the first 2 years of 
the CRTIEC project, much of the work included writing 
storybooks and organizing the supplemental curriculum. 

During the curriculum development process, previous 
research on early literacy interventions for preschool chil- 
dren was explored. Few of the studies targeted children 
thought to benefit from a second tier of support because of 
low literacy skills (Koutsoftas, Harmon, & Gray, 2009). 
Most of the studies focused on classwide interventions that 
included phonological awareness training, typically consid- 
ered Tier | support (Massetti, 2009; Schwanenflugel et al., 
2010; Wilcox, Gray, Guimond, & Lafferty, 2011). Many 
other approaches provided individual instruction, typically 
considered Tier 3 support, with children with or at risk 
for speech and/or language impairments (Bowyer-Crane 
et al., 2008; Hindson et al., 2005; Nancollis, Lawrie, & 
Dodd, 2005; Roth, Troia, Worthington, & Handy, 2006). 
Common themes of interventions that have demonstrated 
efficacy include direct and explicit instruction (Justice, Chow, 
Capellini, Flanigan, & Colton, 2003), instruction at the 
phoneme level (Gillon, 2005; Hesketh, Dima, & Nelson, 
2007; Lundberg, Frost, & Petersen, 1988; Nancollis et al., 
2005; van Kleeck, Gillam, & McFadden, 1998), and in- 
struction that combines phonological awareness and alpha- 
bet knowledge skills (Gillon, 2000, 2002; Justice et al., 2003). 
These principles were incorporated into the lessons that 
were developed for our curriculum. Implementation in a 
small-group format also was supported in previous interven- 
tion studies (Byrne & Fielding-Barnsley, 1995; Justice et al., 
2003; Koutsoftas et al., 2009; O’Connor, Jenkins, Leicester, 
& Slocum, 1993). 

As books were being developed, pilot testing with 
prototypes was conducted in several classrooms to gauge 
children’s interest in the stories and the feasibility of the 
intervention. Teachers helped select students who would 
benefit from additional instruction. Researchers then imple- 
mented some lessons in the classroom with these students, 
and teachers observed and provided feedback. It became 
apparent during this pilot testing that students required 
multiple opportunities to respond and explicit instruction 
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Figure 2. Theory of change for Tier 2 early literacy intervention development. 


Child and Family Characteristics 


awareness 


Language School 
Skills Attendance 


Family 
Demographics 


Linguistic 


Input curriculum 


Home Literacy 
Environment 


Teacher and Classroom 
Characteristics 


Instructional 
Philosophy 


Experience 
and Training 


Number of 
Students with 
Language Delays 


lesson) 


“Business as Usual” Instruction 
Low rates of explicit instruction on phonological 


e Limited exposure to rich linguistic input 

e Few practice opportunities 

e Lack of consistent instructional language 
e = Limited instructional support provided by 


Effective 
Tier 2 
Instruction 


Supplemental Instruction 


High rates of explicit instruction in phonological 
awareness and alphabet knowledge (~10-15 prompts 
per lesson, one new letter/sound per unit) 

e Frequent active academic engagement (multiple 
opportunities to respond per target) 

e Repeated practice opportunities in multiple contexts 
(3 parallel lessons, different prompts and games, 
target words are high-frequency for preschool-age 
children, embedded alphabet knowledge instruction). 

e High fidelity implementation (scripted instruction 
provided by interventionist, audio, or CAI) 

e Teacher-supported practice (review at the end of each 


Curriculum based assessments to monitor progress 


Child Outcomes 


Proximal: Limited growth 
in phonological awareness 
skills; varied performance 
on alphabet knowledge 
skills 

Distal: (not measured) Poor 
early reading skills; 
increased chance for 
reading disability 


Child Outcomes 


Proximal: Growth in 
phonological awareness 
skills (ID first parts and 
sounds of words, 
segmenting, blending) and 
acquisition of letter names 
and sounds 

Distal: (not measured) 
improved reading skills 


on target skills. Lessons were modified on the basis of these 
observations. 

Also during this time, curriculum-based assessments 
(IGDIs) were being developed, evaluated, and scaled using 
item response theory (McConnell et al., 2014). The use 
of IGDIs for identifying students struggling to learn phono- 
logical awareness skills was investigated. The assessments 
also served as a guide for the specific skills that should be 
targeted during intervention. 

During the third year of the CRTIEC project, an 
experimental study was conducted. Instruction targeting 
rhyming, alliteration, and alphabet knowledge was em- 
bedded in the prerecorded storybooks. A multiple-baseline 
across-behaviors research design was implemented with 
nine preschool children. There was a loss of experimental 
control during baseline on measures of alphabet knowledge. 
Five of the nine students demonstrated effects on rhyme 
production, and six of the nine demonstrated improve- 
ments on DIBELS First Sound Fluency. Students did not 
demonstrate improvement on the IGDI curriculum-based 
measures. Because of the inconsistent results, it was difficult 


to determine whether the intervention itself, classroom in- 
struction, or repeated testing were responsible for the gains 
on these skills. 

Several changes were made on the basis of the chil- 
dren’s performance. The lessons were refined to include 
more exemplars of instructional targets. The prerecorded 
stories were adjusted to include shorter response intervals, 
as children seemed to get distracted. It was determined that 
because phonological awareness is a metalinguistic skill, 
instruction should be broad rather than focusing on one or 
two discrete components (Anthony & Lonigan, 2004). Ad- 
ditional phonological awareness skills, including blending 
and segmenting, were added to the instruction. The allitera- 
tion task was changed to a first-sound identification task. 
Frequent, repeated testing was reconsidered because of 
two potential threats to our single-case experimental design. 
First, long baselines of incorrect responding sometimes re- 
sulted in stereotypic responding that children had difficulty 
overcoming once relevant skills were taught. Second, feed- 
back on practice trials may have been sufficient for some 
children to learn phonological awareness skills. 
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Implementation of First Iteration 


A group design study was conducted in the fourth 
year, with 39 students in groups of three randomly assigned 
to either the prerecorded phonological awareness inter- 
vention condition or a “business as usual” comparison con- 
dition. Members of the research team implemented the 
intervention in small groups outside the classroom. Group 
differences were significant for the DIBELS Word Part 
Fluency measure but not the First Sound Fluency measure, 
and the results for the IGDIs curriculum-based assessments 
did not differ significantly between groups. 

It was clear that the effects were not as robust as the 
research team had hoped. Therefore, the intervention was 
re-evaluated and a plan was devised to make modifications. 
We noted that the prerecorded delivery of the instruction 
did not allow for differentiated feedback on children’s re- 
sponses. The lack of contingent feedback seemed to be 
an essential component of instruction, especially with chil- 
dren struggling to acquire this set of metalinguistic skills. 
The effects of this variable were examined in a subsequent 
study. 


Exploration and Preparation of Second Iteration 


A follow-up study was conducted with students who 
did not demonstrate gains during the prior study. The 
selection of children who had not demonstrated progress 
qualified this intervention as a Tier 3 adaptation to the 
intervention (Noe, Spencer, Kruse, & Goldstein, 2014). 
During this study, seven children participated in scripted 
small-group lessons with instruction similar to that which 
was embedded in the previous storybooks; however, stu- 
dents were given feedback on the basis of their responses 
and afforded additional practice opportunities. Researchers 
administered the lessons within the classrooms. During this 
multiple-baseline design study, five of the seven students 
demonstrated gains on the DIBELS First Sound Fluency 
measure following intervention. The results of the Noe 
et al. (2014) Tier 3 intervention with treatment resistors 
were strong enough to guide a major overhaul of our in- 
tervention approach. We concluded that for most children 
with delays in early literacy development, contingent feed- 
back was an essential component and one that could not 
be incorporated within an automated, prerecorded instruc- 
tional format. 

Given the strong results and initial success demon- 
strated in the Ziolkowski and Goldstein (2008) study, we 
were disappointed in our progress. After 4 years and several 
informative studies that produced mediocre results, this 
overhaul was deemed a “do over.” The data suggested that 
we rely on the literature, our collective wisdom, and clinical 
expertise to generate a new supplemental curriculum. For 
the sake of efficiency, we decided to abandon the story- 
books as the context for instruction, and the research team 
worked to develop a set of scripted lessons that were easy 
to deliver and included explicit instruction on a variety 
of phonological awareness and alphabet knowledge skills. 


A scope and sequence was developed to include phonologi- 
cal awareness instruction starting at the whole-word level 
and progressing to the phoneme level across four distinct 
phonological awareness skills: blending, segmenting, first- 
syllable identification, and first-sound identification. 
Consistent with previous research (e.g., van Kleeck et al., 
1998), rhyming was deemed an unnecessary skill for im- 
proving phonemic awareness and early literacy ability in 
preschool-age children; therefore, it was not included in the 
revision of the lessons. 

The instruction included interactive trials, pictures, 
and movements or gestures to increase child engagement. 
Like its prerecorded predecessor, the scripted lessons were 
delivered in small groups during daily 10- to 15-min ses- 
sions. Different types of feedback were included on the 
basis of the groups’ responses for each phonological aware- 
ness trial, thus allowing extra modeling and practice oppor- 
tunities for items that were challenging to one or more 
students. The addition of feedback provides scaffolding for 
students who are learning new phonological awareness 
skills (Schuele & Boudreau, 2008). 


Small-Scale Implementation of Second Iteration 


The new supplemental curriculum, named PA th 
to Literacy, was evaluated in the fifth year of the CRTIEC 
project. An early efficacy study was conducted with groups 
of three students in three Head Start classrooms (Kruse, 
Spencer, Olszewski, & Goldstein, 2015). Members of the 
research staff administered the intervention outside the 
classroom. Students were identified as intervention candi- 
dates using a combination of IGDI and DIBELS screen- 
ing measures. A multiple-baseline design across groups 
of participants was used to examine the effects of PAth to 
Literacy on phonological awareness skills of students. 
Seven students completed the intervention. DIBELS First 
Sound Fluency, Word Part Fluency, and Letter Naming 
Fluency served as the primary outcome measures for this 
study. 

The results of the early efficacy study demonstrated 
that all students showed significant gains on the First Sound 
Fluency measure and maintained those gains. During 
baseline, experimental control was lost for two students on 
the Word Part Fluency measures, but only one child ever 
scored above 4 on First Sound Fluency. Following interven- 
tion, all students consistently scored above the benchmark 
for the beginning of the kindergarten year (10). Generalized 
effects were demonstrated across a number of phonological 
awareness measures, although average gains on the TOPEL 
phonological awareness and print knowledge subtests were 
not robust. This is likely due to the fact that many of the 
tasks in the TOPEL do not align with the skills taught in 
PAth to Literacy. The experimental results for this scripted 
intervention seemed strong enough to warrant a larger scale 
study with classroom staff administering the intervention, 
but factors related to implementation also needed to be 
considered. 
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Preparation of Large-Scale Implementation 
of Second Iteration 


Two sources of information helped inform our prepa- 
ration for implementation by teaching staff in classrooms. 
First, we took advantage of opportunities to solicit feed- 
back from teachers who were interested in implementing 
our curriculum materials. Although researchers delivered 
the intervention in our prior studies, social validity assess- 
ments were gathered from classroom staff, who were inter- 
viewed and asked to complete brief, five-item Likert-type 
surveys to determine how satisfied they were with the PAth 
to Literacy curriculum. For example, in the Kruse et al. 
(2015) study, all three classroom teachers indicated that 
they felt that they would be able to incorporate the lessons 
into their classroom routines. The average score on the con- 
sumer satisfaction surveys was 5.4 (1 = very unsatisfied, 

6 = very satisfied), indicating that teachers were satisfied 
with the intervention. 

Second, after the Kruse et al. (2015) study, we ac- 
tively sought assistance from teachers who had worked 
with us previously. Four teachers were recruited to practice 
implementing the lessons in their classroom to determine 
feasibility and training needs. These teachers were provided 
training materials including a manual, sample videos, and 
sample lessons. They also were provided two brief one-on- 
one training sessions with a member of the research staff 
to learn how to implement PAth to Literacy. These teachers 
were subsequently observed implementing the interven- 
tion on three separate occasions to determine the extent to 
which they could meet our criteria for implementation fidel- 
ity after minimal training. Although the training sessions 
were relatively brief (less than | hr), teachers administered 
the lessons with an average of 77% fidelity. This indicated 
that with extended training and regular coaching from the 
research staff, teachers would likely be able to implement 
the intervention in their classrooms with high fidelity. 


Large-Scale Implementation of Second Iteration 


Once efficacy and feasibility were established, the 
CRTIEC team organized an effectiveness trial of PAth to 
Literacy during the final year of funding. This multisite 
cluster-randomized design trial was conducted in 39 urban 
classrooms in Florida, Kansas, and Ohio. In Florida, 

20 child care providers that delivered state-funded volun- 
tary pre-K services participated in the study. In Kansas, 

11 public preschool classrooms were recruited, and eight 
public preschool classrooms were recruited in Ohio. All 
teachers participated in an initial large-group training that 
lasted 2 to 3 hr. Teachers also received a comprehensive 
training manual and two training videos that they could use 
to prepare for delivering the intervention. Prior to the start 
of the intervention, a member of the research team visited 
the teacher in the classroom to complete a “checkout” pro- 
cedure that included delivering part of a lesson to ensure 
that teachers understood the intervention and had a plan 
for integrating instruction into their daily routine. 


Multiple waves of screening using DIBELS and IGDI 
screening methods and the TOPEL phonological awareness 
subtest were used to identify two or three students in each 
classroom who qualified for the intervention. It was im- 
portant to conduct multiple waves of screening throughout 
the fall semester to help identify students with delays who 
were not demonstrating growth on phonological awareness 
tasks despite several months of Tier | (general classroom) 
instruction. Once researchers were certain that each class- 
room would have at least two or three candidates, classrooms 
were randomized to the PAth to Literacy intervention or 
to a comparison vocabulary intervention, Story Friends: 
Jungle Friends. Thus, all children received small-group 
instruction on the same consistent basis, providing a strong 
test of intervention effects rather than a business-as-usual 
control. 

The intervention phase lasted 24 to 36 sessions de- 
pending on students’ performance on lessons. Each lesson 
lasted approximately 10 to 15 min. Teachers or instruc- 
tional assistants delivered lessons in the classroom. Mem- 
bers of the research staff provided weekly coaching support 
and completed measures of implementation fidelity. Over- 
all, implementation fidelity was high, averaging 84% on 
an observational tool. Teachers responded favorably to 
weekly coaching, and most demonstrated improvement in 
fidelity scores as they progressed through the intervention. 

A total of 113 students participated in this interven- 
tion. Results indicated that there were significant group 
differences on the DIBELS First Sound Fluency and Word 
Part Fluency measures and large effect sizes. Both groups 
demonstrated gains on the phonological awareness and 
print knowledge subtests of the TOPEL, although group 
differences did not reach significance. Among children who 
demonstrated delays in early literacy development moni- 
tored over the course of 4 months of instruction in their 
prekindergarten year, a large difference was shown in those 
meeting the benchmark for the beginning of kindergarten 
on the DIBELS First Sound Fluency test (81% vs. 34% 
in the experimental and comparison groups, respectively). 
Overall, these results indicated that teachers were able to ef- 
fectively implement PAth to Literacy in the classroom, and 
students seemed to demonstrate relatively robust improve- 
ments on the targeted phonological awareness skills. 

Following the intervention, teachers were asked to 
complete satisfaction surveys regarding their experience 
with the intervention in their classroom. The survey in- 
cluded 22 positive statements about the intervention and 
asked teachers whether they agreed with each statement 
on a Likert-type scale (1 = strongly disagree, 6 = strongly 
agree). Overall, teachers agreed with the statements (7 = 
4.20-5.07). Teachers were most satisfied with the adequacy 
of training and ease of lesson delivery. Teachers in Florida 
and Ohio rated the intervention more positively than did 
teachers in Kansas. The less-satisfied teachers noted that 
the amount of time required to deliver the lessons was 
burdensome and that it was difficult to engage students dur- 
ing some lessons. A focus group also was held to discuss 
changes the teachers recommended for the intervention. 
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Teachers in the focus group were satisfied with the results 
of the intervention but noted that it was sometimes difficult 
to manage the lesson materials. They had specific sugges- 
tions about units that required more effort to administer. 
They noted that it was initially difficult to focus on a small 
group of students at one time to provide appropriate feed- 
back. The teachers in the focus group expressed interest 

in using the PAth to Literacy lessons in the future. Future 
iterations of PAth to Literacy will strive to integrate this 
teacher input. 


Implementation Considerations 
Exploration Phase 


The intervention research described above corresponds 
mainly to the exploration and preparation phases of imple- 
mentation. Aarons et al. (2011) suggest that there is little 
literature about the exploration phase because most of the 
literature focuses on the adoption of innovation. Perhaps 
because the federal funding agency required the CRTIEC 
team to play a leadership role, a good deal of attention 
went into this phase of implementation. For example, the 
CRTIEC team monitored changes in stages of implementa- 
tion of RTI across the United States from 2009 to 2012 
(Greenwood et al., 2014). Networks were formed through 
the CRTIEC website and an annual RTI in Early Child- 
hood Summit. These networks discussed advocacy among 
professional organizations, which eventually resulted in a 
joint position statement (Pretti-Frontezak et al., 2014) that 
served to promote understanding and provide guidance 
for RTI frameworks in early childhood contexts. This 
proactive attention to outer context factors has helped 
set the stage and speed up the process for later phases of 
implementation. 

Partnerships formed between the CRTIEC team and 
an array of early childhood education agencies helped to 
guide considerations of the inner context during the explo- 
ration phase. Multiple program structures were considered 
(e.g., Head Start, Title 1, public pre-K, subsidized child 
care, voluntary pre-K). Not all classrooms were ready to 
embrace change. For example, some programs that did not 
have a defined classroom schedule that included some type 
of center rotation incorporating explicit instruction were 
deemed poor candidates for implementing an RTI system. 
This did not disqualify many classrooms because a center 
rotation was a common practice. From its inception, Tier 2 
early literacy intervention was designed to fit within that 
scheduled activity. Thus, the intervention was developed 
for daily small-group instruction during 10- to 15-min ses- 
sions. The intervention teaches skills that are widely recog- 
nized as being important for school readiness (phonological 
awareness and alphabet knowledge; National Early Liter- 
acy Panel, 2008). Programs that believed this goal was not 
developmentally appropriate were not good candidates 
for implementation. However, such beliefs were infrequent, 
as administrative leaders, at least, were aware that the tar- 
geted skills were included in kindergarten readiness testing 


in their states. In addition, there were state and national 
initiatives designed to increase reading skills in all children 
starting at an early age (e.g., Early Reading First). These 
interconnections between inner and outer contexts during 
the exploration phase facilitated movement into the prepara- 
tion phase of implementation. 


Preparation Phase 


The preparation phase is when adoption decisions are 
made. Aarons et al. (2011) pointed out that adoption deci- 
sions are not a one-time event because organizations often 
choose to try out an innovation prior to broader imple- 
mentation. Moreover, the CRTIEC staff did not have the 
resources to scale up implementation within large urban 
districts. However, we deliberately sought long-term part- 
nerships whenever possible. This meant that we often 
worked in the same classrooms but with different cohorts 
of students; however, administrators moved us around to 
meet their needs as well. Administrators were agreeable 
to having research staff serve as the interventionists initially 
to ensure fidelity of implementation and demonstrate that 
the intervention was worthy of adoption. Because of the 
understanding that future scale-up efforts would involve 
teachers as interventionists, we sought their impressions of 
the intervention and what they would need to be able to 
carry out the intervention easily in their classrooms. Results 
were shared with administrators over the summer and with 
classroom staff (along with in-service training) each fall. In 
some cases, administrators were willing to evaluate results 
on kindergarten readiness tests, which went beyond the 
scope of our approved studies. This initiative on the part of 
the administrators was welcomed, however. They saw re- 
sults that convinced them that students were benefiting 
from our involvement. That and the willingness of a research 
team to work with them year after year helped them become 
champions for adoption, which facilitated the next stage 
of implementation. The attention and resources that went 
into planning and development activities helped prepare 
sites for active implementation. 


Active Implementation Phase 


We have accomplished the early steps of active im- 
plementation in districts in Florida, Ohio, and Kansas. 
Teachers and aides from 39 classrooms were taught to im- 
plement early literacy or language interventions. This might 
be considered small-scale systems changes, as it represents 
a small percentage of the classrooms in these early child- 
hood programs. This scale up was advisable as an initial step 
in evaluating the effectiveness of PAth to Literacy in authen- 
tic pre-K contexts. The results are being used to inform 
continued improvement of the intervention as well as the 
implementation process. 

The outer context of statewide testing served as an 
important consideration in the implementation process. 
Because voluntary pre-K funding in Florida is contingent 
upon classwide performance on kindergarten readiness 
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testing (including phonological awareness skills), teachers 
were very motivated to adopt the PAth to Literacy cur- 
riculum. In several schools, teachers used the lesson mate- 
rials with students in the class who had not participated 

in the intervention, and many teachers asked to keep the 
intervention materials following the conclusion of the 
study. The teachers in Kansas and Ohio were certified 
teachers with bachelor’s degrees (or higher), whereas in 
Florida child care providers often lacked educational train- 
ing and teaching certification. Differences in education or 
experience may have played a role in teachers’ willingness 
to adopt new ideas and classroom interventions. 

Inner contexts such as individual teacher and class- 
room differences were accounted for as well. Some teachers 
required frequent coaching support from the research staff. 
Other teachers delivered the lessons independently early on, 
and researcher interaction was limited to observing fidelity 
of implementation. Student behavior accounted for some of 
the variability in the amount of teacher support provided 
by the researchers. Several of the classrooms contained stu- 
dents who frequently engaged in challenging behavior, and 
the research staff problem solved behavior management 
and positive support strategies to implement in these class- 
rooms. The number of staff in each classroom also contrib- 
uted to the amount of support that teachers required. 

Although inner and outer contexts were considered 
in the implementation phase of PAth to Literacy, several 
considerations of the EPIS model may have increased the 
effectiveness of the intervention. At the level of outer con- 
texts, organizational networks should have been considered. 
In Florida, the voluntary pre-K classrooms received sup- 
port from the district’s school readiness office. Although 
the school readiness staff was supportive in recruiting class- 
rooms, they were not sufficiently trained in the interven- 
tions or included during the implementation phase of the 
study to support sustainability efforts. For more effective 
implementation and sustainment, the district staff should 
be able to provide more support throughout the process, 
thus allowing the district to take more ownership over the 
intervention. Networks within the district also could be put 
into place to allow teachers to connect and share experi- 
ences during the implementation process. With more fore- 
thought, the research team may have been able to involve 
the district staff to a greater degree. 

At the inner context level, individual teacher charac- 
teristics should have been considered further. In particular, 
teacher willingness to change or adopt new curricula should 
have been gauged more carefully prior to implementation. 
Alignment of interventions with school and teacher philoso- 
phy is important for sustained implementation within class- 
rooms (Forman, Olin, Hoagwood, Crowe, & Saka, 2009). 
As mentioned before, teachers differed in their willingness 
to implement the interventions. Although all teachers pro- 
vided informed consent and voluntarily agreed to partic- 
ipate in the study, it was clear that some felt it was an 
obligation whereas others utilized the study as a learning 
opportunity. In one particular instance, a teacher discon- 
tinued delivery of the lessons after the first 2 weeks of the 


study. She stated that she was overwhelmed with her re- 
sponsibilities and did not have time to implement the lessons. 
She agreed to allow a volunteer provided by the research 
team to deliver the lessons in her classroom. Following the 
study, she expressed satisfaction with the lessons and the 
progress of her students and said that she would like to im- 
plement the lessons in the future. Although this teacher 
changed her attitude through the course of the study, it is 
clear that she was not willing to adopt a new intervention 
at the start. Developing relationships and perhaps probing 
further into each teacher’s willingness to change prior to 
the initiation of the study may be needed to reduce barriers 
in implementation. It was interesting to see how attitudes 
can change over time. Many of the teachers in Ohio and 
Kansas had seen the interventions being implemented by 
the research team with their students. For many this raised 
their confidence in their ability to implement interven- 
tions they had observed, whereas others were resentful that 
they were being asked to add this instruction to their daily 
activities. 


Sustainment Phase 


Our team currently is focused on scaling up active im- 
plementation and moving into the sustainment phase of 
PAth to Literacy. Several teachers who participated in our 
study kept materials with the intention of continuing their 
use in classrooms. The school readiness program associated 
with the school district in Florida has expressed interest 
in utilizing the materials in all classrooms. For this to be 
feasible, a training and support team would need to be put 
into place. One obstacle to districtwide implementation 
is funding. Preschools, child care centers, and districts are 
limited in the funding they can use to purchase new mate- 
rials. In retrospect, it would have been wise to involve 
higher administrative personnel in discussions and data 
sharing earlier, as the plans for allocating funds for curric- 
ula and for professional development often are made | year 
in advance. 

Once funding has been secured, districts and organi- 
zations will need a system of training and coaching to sup- 
port teachers implementing the intervention. This likely will 
be accomplished by training support staff at the district 
or organizational level or by teachers who volunteer to be 
training leaders. Members of the research team who helped 
develop the intervention should serve as leaders in the ini- 
tial training phase, with a gradual shift of leadership to the 
organizational and classroom levels. 

In addition to initial training, sustaining implementa- 
tion will require ongoing coaching or other tactics (Forman 
et al., 2009). Some assistance may be provided through an 
online community of support that may include frequently 
asked questions, message boards, videos, and supplemental 
documents. This would allow teachers to access informa- 
tion and interact independently. Ongoing monitoring of 
implementation fidelity is important to ensure that students 
are receiving the supplemental curriculum as it was de- 
signed to be delivered (McKenna, Flower, & Ciullo, 2014). 
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Teachers also may require support integrating the inter- 
vention into their daily routines (Cornett & Knight, 2009). 
District training leaders and site coaches may be responsi- 
ble for resolving scheduling conflicts. Further research 
should explore ways in which teachers can be fully supported 
in delivering interventions. 

How to situate phonological awareness intervention 
within the larger context of a programwide RTI model also 
needs to be investigated. The PAth to Literacy team has 
partnered with a group of researchers who specialize in be- 
havior and social-emotional support strategies using an 
RTI (pyramid model) system for preschool-age students 
(Fox, Carta, Strain, Dunlap, & Hemmeter, 2010). An effort 
is currently under way to examine ways in which an inte- 
grated RTI model can be effectively developed and imple- 
mented within preschool classrooms. Although interventions 
addressing behavior, language, and literacy have been effec- 
tive in stand-alone studies, it is important to explore how 
to integrate these interventions in classroom contexts (Ball & 
Trammell, 2011). 

One issue that arises with integrating the supplemen- 
tal curriculum into an RTI model is the reliance on regular 
screening and progress monitoring assessments. In the im- 
plementation studies described, all testing was conducted 
by members of the research staff. Teachers are likely to re- 
quire training to learn how to administer the measures and 
make decisions regarding assigning students to differentiated 
tiers of instruction. In addition, when the phonological 
awareness intervention was modified, the curriculum-based 
measures (IGDIs) were not modified accordingly. This may 
have resulted in a misalignment of testing and instruction, 
which could explain the lack of robust results on the IGDIs 
during the cluster randomized trial of PAth to Literacy. 
Future work will further develop curriculum-based mea- 
sures to monitor student growth on discrete skills taught in 
the supplemental curriculum. 


Conclusions 


Even though implementation of PAth to Literacy was 
not specifically guided by the EPIS model, it is clear that 
many of the principles aligned. There were several strengths 
in our implementation process. Perhaps the most important 
component of this journey was the constant use of data- 
based decision making. From pilot testing to single-subject 
experimental design studies to larger group design studies, 
each step of the intervention development process relied 
on collecting data and making adjustments to the interven- 
tion as necessary. This may make a difference between a 
mediocre and a potent evidence-based practice. It is impor- 
tant that continued active implementation and sustainment 
phases of implementation capture information that also 
can be used to help improve and refine the intervention. 

Feasibility of implementation was considered consis- 
tently from the exploration phase on. The researchers un- 
derstood many of the challenges to delivery of RTI services 
in early childhood settings and anticipated some of the im- 
plementation difficulties. Factors such as low cost, ease of 


delivery, and short duration of intervention were important 
considerations from the start. Teacher feedback was utilized 
during all implementation trials so that appropriate ad- 
justments could be made to improve feasibility. However, 
social validity assessments that occur only at the end of 
studies may not be the best means of getting honest feedback 
(Goldstein, in press; Schwartz & Baer, 1991). The research 
team had the benefit of informal conversations because 

of frequent visits to classrooms. We discussed these conver- 
sations with teaching staff in our weekly lab meetings be- 
cause they often were important sources of information about 
teachers’ perceptions of the intervention. 

We found ourselves abandoning an intervention for- 
mat (prerecorded lessons embedded in storybooks) that 
had enjoyed initial success, delivered in a manner that was 
highly acceptable to teaching staff, because of less-than- 
stellar results. This illustrates the nonlinearity that can 
occur in intervention development and implementation pro- 
cesses. It unfortunately took three expensive studies (only 
one of which was published) before we made the decision 
to overhaul the intervention. Nevertheless, this tradition 
of following the data and creative problem solving is what 
science is all about. Although PAth to Literacy has demon- 
strated efficacy and effectiveness, the research team is con- 
tinuing to explore, plan, and implement modifications to 
the lessons. This refinement process allows researchers 
to strive for more effective and efficient means of delivering 
services. The process also helped the research team under- 
stand that different media were effective for teaching dif- 
ferent skills. For example, prerecorded storybooks were 
effective for teaching vocabulary (Kelley & Goldstein, 2015; 
Spencer et al., 2012) but not for teaching phonological 
awareness skills. 

The federal funding of a multisite collaborative agree- 
ment afforded the developers the unique experience of gain- 
ing insight from researchers with varied areas of expertise 
as well as from teachers and students from different geo- 
graphic settings. Effects of the intervention were replicated 
across three separate locations under the supervision of 
separate but coordinated research teams. In addition, 
CRTIEC team members contributed their own unique ex- 
periences and expertise to the project. The simultaneous 
development of assessment and multiple interventions was 
beneficial in understanding how each specific intervention 
fit within the larger context of early childhood. 

Last, the gradual release of control from the interven- 
tion developers to the classroom teachers was important 
for implementation. This afforded researchers firsthand ex- 
perience that continues to guide future development of the 
intervention with feedback from teachers. It also allowed 
researchers to closely observe implementation of PAth to 
Literacy within classrooms by trained teachers and their 
instructional assistants. Specific training and support needs 
were identified. For example, we realized that paraprofes- 
sionals who initially may have been hesitant often excelled 
in delivering the scripted lessons in small groups. The close 
partnership of the intervention developers and the classroom 
teachers allowed for a collaborative development process. 
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Although the implementation of PAth to Literacy 
has experienced initial success, attention to additional outer 
and inner context factors is needed to continue scaling up 
this Tier 2 intervention, to explore the breadth of applica- 
bility (e.g., kindergarten), and to promote sustainability 
across different early childhood settings. 

Approximately 10 years of intervention development 
and evaluation provided an ever-changing research team 
intimate knowledge of many facets of early literacy. It is 
through many years of research that one gets to know a be- 
havioral phenomenon. These experiences reinforced the 
need to not be easily satisfied, as we continue to seek better 
measurement schemes and more potent intervention proce- 
dures. We are aware that abandoning an automated inter- 
vention format in favor of a scripted intervention presents 
new challenges. We have learned that implementation is 
feasible and results appear to be robust, but it will require 
more effort to maintain sustainability with high fidelity. 
The process of creating a supplemental curriculum that was 
effective and practical may seem analogous to the adage 
about “building the plane while flying it.” We contend that 
researchers will do the field a disservice if they do not balance 
the processes of intervention and implementation develop- 
ment. Like much of the behavioral sciences, communication 
sciences and disorders have too few evidence-based prac- 
tices that have undergone years of iterative development 
and refinement to the point of producing optimal, well- 
defined intervention protocols. By considering implementa- 
tion and sustainment during the intervention development 
process and using data-based decision making, the field will 
hopefully expand its repertoire of user-friendly evidence- 
based practices and encourage a bridging of the researcher— 
clinician gap. 
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